In video image encoding, each image included in a video image and corresponding to multiple time points is divided into multiple blocks, and an encoding process is executed on the blocks on a block basis. For example, in High Efficiency Video Coding (HEVC) that is one video image encoding scheme, blocks of a layered structure that are coding units (CUs), prediction units (PUs), transform units (TU), and the like are used. The size of a block of n pixels×n pixels (n is an integer of 1 or more) is hereinafter referred to as n×n in some cases.
As block sizes of the CUs, PUs, and TUs, the following sizes may be selected, for example.
The CUs: 8×8 to 64×64
The PUs: 4× 4 to 64× 64
The TUs: 4×4 to 32×32
A video image encoding device configured to control an upper limit frequency or orthogonal transform size of an orthogonal transform coefficient based on the variance of pixel values of an input image is also known (refer to, for example, Japanese Laid-open Patent Publication No. 2003-259376 and Japanese Laid-open Patent Publication No. 2008-219205).